1. Field of the Invention
The present invention relates generally to device testing processes, and more particularly, to methods and apparatus for developing testing systems for comparison testing of two or more similar electrical devices.
2. Description of the Related Art
Development of a new device such as an integrated circuit, or other electronic device, requires extensive testing to determine if a proposed design meets the design performance requirements. One approach to developing a new device includes simulating the device such as by designing a computer program that simulates the operation of the device. Such a computer program is typically referred to as a device simulation. Device simulations are typically run in a test bench environment. Multiple test procedures or test cases are then applied to a device simulation running on the test bench. A test suite is a collection of test cases that test multiple functions of a simulated device.
Typically each test case is in the form of computer readable code executable by the test bench. A test case causes the test bench to apply stimuli to the device simulation and record the simulated device operation. Various functions of the simulated device can then be tested. Simulated device operations can include input, output, processing, events, signal levels, and any other device function.
Test cases are often written using object oriented programming languages such as C++, Java(trademark), Vera and other languages. Generally speaking, methods of a test object are designed to invoke the methods of a number of objects, thus causing very specific stimuli to be applied the device simulation.
Accordingly, the code programmed for each test case is specifically designed for each device to be tested. In practice, a test case for a device (e.g., a second-generation of the device) is typically developed by first selecting a similar device (e.g., a previous generation of the device) that is most similar to the device. A first test case for the similar device is selected because the first test case presumably, successfully tested the similar device and is therefore believed to be accurate, comprehensive and properly debugged. The first test case is then modified to test the specific function of the device to be tested.
As a result, the test cases often become difficult to determine precisely what functions of the device are being tested. Further, as the test cases become successively modified, the performance data can become more difficult to obtain and less accurate and less comprehensive. Accurate and comprehensive performance data is important so that a first device under test can be directly and accurately compared to another device under test (e.g., a second-generation device).
In addition, developing a new test suite for a second-generation device upon the test suite developed for a first generation device typically requires extensive man-hours to modify and debug. Further, if a fault is found in the new test suite and the fault is corrected during the testing or development of the second-generation device, then the fault may also apply to the first generation device. Therefore the test suite for the first generation device may also require a corresponding correction.
A test suite may include many hundreds of test cases that are similar. When a defect is found in a selected test case, often most or all of the similar test cases must also be corrected. Therefore, even a minor defect discovered in one test case in the new test suite, often results in extensive efforts to update each of the similar test cases in the new test suite and in any similar test suites.
Device manufacturers are under constant pressure to reduce the development time for new devices. One area of reducing the development time is reducing the time required to test various proposed, and often competing configurations of the device. As discussed above, the prior art approach to testing a new device was to modify an existing test object to generate a modified test case, which can then be applied to the new device. However, the administrative management requirements of the prior art approach cause additional delays in device development time.
Another area for reducing the development time is an evaluation of multiple configurations or designs of a device at an early stage in the development cycle. For example, if two competing designs are proposed for a new product, each of the competing designs must be evaluated to select the design to carry forward into production. Often, in the prior art, both of the competing designs must be carried through to the hardware stage so that the two designs could then be tested as actual hardware circuits (e.g., integrated circuits) installed in a host system, such as a host computer system, and the resulting performances compared. The two designs cannot typically be readily tested and compared prior to the hardware stage because the prior art test cases typically do not produce accurate performance data that can be directly compared.
In view of the foregoing, there is a need for a system and method for rapidly generating test cases that produce accurate, comprehensive test results and that allow multiple configurations of a device to be accurately and directly.
Broadly speaking, the present invention fills these needs by providing a system and method for comparing performance of two or more devices is disclosed. The method includes defining a workload. A workload object is provided and the workload object includes a transaction generator. A first test case is applied to a first device. The first test case can be produced by executing the transaction generator on a first test object. The first test object can include multiple, first interface properties. Executing the transaction generator on the first test object can produce a first set of transactions according to the first interface properties. The first set of transactions can stimulate the defined workload in the first device. A second test case is applied to a second device. The second test case can be produced by executing the transaction generator on a second test object. The second test object can include multiple second interface properties. Executing the transaction generator on the second test object can produce a second set of transactions according to the second interface properties. The second set of transactions can stimulate the defined workload in the second device. A first test result from the first test case can be compared to a corresponding second test result from the second test case.
In another embodiment, the first interface properties are included in a first interface object. The first test object can inherit the first interface object and the workload object.
In another embodiment, the first interface object can include several interface objects. The interface objects can include a host interface object and a peripheral interface object.
In one embodiment, the interface objects can include a parameter object. The parameter object can define a parameter that corresponds to a parameter of the first device.
In one embodiment, the second interface properties can be included in a second interface object. The second test object can inherit the second interface object and the workload object.
In one embodiment, the defined workload is generic to several members of a device family.
In one embodiment, the first set of transactions includes at least one of a group consisting of a read, a write, an event and an interrupt. In one embodiment, the second set of transactions includes at least one of a group consisting of a read, a write, an event and an interrupt.
In still another embodiment, a test bench is disclosed. The test bench includes a processor and a storage facility coupled to the processor. The storage facility includes instructions executable by the processor. The instructions include logic to receive a workload and logic to provide a workload object. The workload object can include a transaction generator. The instructions can also include logic to apply a first test case to a first device. Executing the transaction generator on a first test object can produce the first test case. The first test object can include multiple first interface properties. Executing the transaction generator on the first test object can produce a first set of transactions according to the first interface properties. The first set of transactions can stimulate the defined workload in the first device. The instructions can also include logic to apply a second test case to a second device. Executing the transaction generator on a second test object can produce the second test case. The second test object can include multiple second interface properties. Executing the transaction generator on the second test object can produce a second set of transactions according to the second interface properties. The second set of transactions can stimulate the defined workload in the second device. The instructions can also include logic to compare a first test result from the first test case to a corresponding second test result from the second test case.